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HOW  THE  U.S.  BUYS  RESEARCH 

When  you  compete  for  a  government  R&D  contract, 
what  are  your  chances  of  winning? 
We  know  there  is  a  formal  contract  award  process, 
but  does  it  really  mean  anything? 

by  Edward  B.  Roberts 


In  Brief:  There  is  a  formal  contract  aivard 
process  that  supposedly  initiates  all  U.S. 
government  research  funding.  But  there 
are  major  differences  between  this  formal 
process  and  the  actual  process  by  which  the 
government  buys  research.  The  author 
analyzes  these  differences  and  shows  that 
the  actual  process  is  far  less  competitive 
than  government  regulations  would  suggest 
it  to  be.  In  fact,  there  is  a  good  deal  of 
superficial  competition. 

In  this  analysis  of  the  muUibillion-dollar 
contracting  business,  the  author  .stiggests 
how  the  research  prociirement  process 
ought  to  be  changed  to  rid  itself  of  super- 
ficial competition  and  the  wastefulness  it 
causes.  Your  review  of  this  document 
should  increase  your  understanding  of  the. 
actual  process  .  .  .  and  it  may  explain  why 
you  failed  to  get  that  last  contract. — D.A. 

■  How  does  the  U.S.  government  buy  research 
and  development?  We  know  it  awards  over 
seven  billion  dollars  per  year  to  industrial 
firms,  plus  another  billion  and  a  half  to  non- 
profit corporations  and  universities.  And  we 
know  there  is  a  formal  contract-award  process 
that  supposedly  initiates  all  this  funding.  I 
have  charted  this  formal  process  on  the  op- 
posite page — and  it  does  indeed  look  formal, 
even  formidable. 

But  does  it  really  work  this  way?  Is  it  really 
a  filtering  process,  as  this  sketch  suggests? 

From  a  series  of  management-research  in- 
quiries I  have  made,  under  general  auspices 
of  the  NASA-sponsored  MIT  Organization 
Research  ProKram,  I  have  come  to  the  con- 
clusion that  actual  operation  of  the  contract- 
award  process  does  not  correspond  to  that 
which  one  would  expect  from  the  formal  regu- 
lations. In  fact,  I  would  go  further  and  say 
that  many  aspects  of  the  formal  contracting 
system  succeed  only  in  increasing  R&D  costs, 
adding  time  delays,  and  producing  other  dam- 
aging effects  on  government-sponsored  re- 
search and  development. 

In  this  article,  I  want  to  explain  how  the 
process  really  works.  In  doing  this,  I  will 
compare  this  actual  process  with  the  formal 
process,  so  let  me  begin  with  a  quick  explana- 


tion of  the  rules  and  regulations  that  are 
supposed  to  dominate  government-sponsored 
R&D.  These  rules  and  regulations  are  incor- 
porated in  the  Armed  Services  Procurement 
Regulations.  With  only  minor  exceptions,  the 
general  principles  of  ASPR  have  been 
adopted  by  most  agencies  of  the  government, 
including  NASA.  In  brief,  the  formal  steps 
work  like  this:  A  government  scientist  or 
engineer  files  a  Procurement  Request  for 
R&D  services  or  equipment.  In  his  request, 
he  includes  some  specifications  of  what  he 
wants  to  have  "researched  and/or  developed," 
plus  when,  why,  how,  etc.  .  .  .  and  he  indicates 
what  companies  might  be  able  to  meet  those 
specifications.  Then  the  request  goes  through 
several  stages  of  approval — financial,  contract- 
ing, headquarters,  legal — and  during  this 
process,  the  Facility  Source  Oflice  at  the  gov- 
ernment field  center  adds  the  names  of  other 
possible  contractors  to  those  recommended  by 
the  scientist  or  engineer  who  initiated  the  re- 
quest. Next,  evaluation  teams  are  appointed, 
evaluation  criteria  established,  and  Requests 
For  Proposals  are  sent  to  companies  who  have 
made  the  list.  Responding  to  these  requests — 
or  noting  the  announcement  of  the  pending 
procurement  in  the  Commerce  Department's 
daily  newspaper — interested  contractors  sub- 
mit their  proposals.  And  finally,  these  pro- 
posals are  evaluated,  award  recommendations 
are  made,  and  the  selected  firm  receives  a 
contract. 

Examining  the  letter  of  these  government 
procedures,  one  ,sees  the  principles  that  the 
formal  process  is  intended  to  promote: 

•  maximum  competition 

•  objective  evaluation — through  numerical 
proposal  evaluation  by  teams  of  evaluators 

•  independent,  multilevel  review. 

But  one  conclusion  I  have  reached,  from 
over  six  years  of  research  on  the  government 
R&D  process,  is  that  these  sought-after  prin- 
ciples are  not  secured.  Even  more  general  is 
my  conclusion  that  many  government  policies 
for  managing  contracted  research  and  devel- 
opment are  in  major  conflict  with  the  govern- 
ment's own  objectives. 

My  study  of  contract  awards  was  made  in 
two  government  agencies — one  was  a  Defense 
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THE  FORMAL  CONTRACT 
AWARD  PROCESS 


Technical  initiator 
prepares  Procurement 
Request,  including 
recommended  sources 


Budgeting  and  funds 
commitment  made  by 
technical  center 


Procurement  Office 
assigns  Contracting 
Officer  and  Contract 
Negotiator 

Facility  Source  Office 
adds  recommendations  to 
the  potential  source  list 


Negotiator  arranges        i 
for  Source  Evaluation 
Board  and  advisory 
committees  ' 

Negotiator  prepares 
Request  for  Findings 
and  Determination  and 
procurement  plan 

Approvals  obtained  from  Legal ' 
Department  and  Headquarters 


Negotiator  issues 
Requests  for  Proposals 
to  source  list  and 
informs  industry 

Contractors  prepare 
and  submit  technical 
and  cost  proposals 


Boards  decide 
proposal  'evaluation 
critieria 


Negotiator  arranges 
a  Bidder^  Conference 


Committees  and  board  evaluate  proposals 
and  make  tentative  recommendations 


Negotiator  arranges 
for  oral  presentations 
by  contractors 


Evaluation  board 
suggestions  are 
presented  to  center 
director  for  approval 


'■ '     Contractor  is 
selected  and 
Negotiator  prepares 
Letter  Contract 

Negotiator  handles  , 

debriefing  sessions 
for  losing  bidders 

The  flow  of  formal  activities  in  a  typical  rcaearck 
and  devrlnpmcnt  contract  award  is  supposed  to 
work  like  this.  The  white  boxes,  in  most  instances, 
apply  only  to  large  contracts  (exceeding  $1  mil- 
lion). For  things  as  they  really  are,  see  text. 


Department  center,  where  we  examined  41 
DOD  awards,  from  $100  thousand  to  $8  mil- 
lion; the  other  was  a  nondefense  agency  and 
here  we  examined  10  contracts,  from  $1  million 
to  $40  million.  (Another  100  awards  are  now 
under  study  in  a  third  government  center.) 
I  want  to  defer  here  to  say  that  I  was  aided 
by  Laurence  B.  Berger  and  J.  Barry  Sloat, 
graduate  students  in  the  MIT  Sloan  School  of 
Management,  and  that  we  were  given  excel- 
lent cooperation  in  both  government  centers: 
We  had  complete  access  to  the  contracting 
files  and  personnel,  and  I  felt  throughout  our 
extensive  interviews  that  we  were  receiving 
honest — not  just  "cooperative" — answers.  In 
the  DOD  office,  we  interviewed  contract  ne- 
gotiators and  some  of  the  technical  initiators 
and  project  managers.  In  the  other  agency. 
we  interviewed  the  chairmen  of  the  source 
evaluation  boards,  members  of  the  technical 
evaluation  committees  and  the  business  eval- 
uation committees,  and  the  contract  negotia- 
tors, project  initiators,  project  managers,  plus 
others  who  were  closely  related  to  project 
award  decisions. 

The  competition 

What  did  we  find  out? 

First,  our  file  searches  suggest  that  about 
60%  of  the  R&D  awards  were  made  on  a  sole 
source  basis — without  formal  competition. 
This  finding  jibed  with  what  M.  J.  Peck  and 
F.  M.  Scherer  had  observed  a  couple  of  years 
ago,  in  their  analysis  of  the  weapons  acquisi- 
tion process;  "In  fiscal  year  1959,  some  53.8% 
by  dollar  volume  of  the  $15.3  billion  in  domes- 
tic military  contract  awards  .  .  .  were  nego- 
tiated noncompetitively  with  a  single  firm." 

Now  let  me  point  out  that  we  excluded  such 
noncompetitive  contracts  from  those  contract 
awards  we  studied.  We  did  this  because  we 
wanted  to  know  how  much  real  competition 
existed  in  that  other  40%— in  that  segment 
of  the  R&D  business  that  got  done  after  com- 
petitive solicitation. 

What  did  we  find  out  here? 
We  found  less  actual  competition  than  one 
might  have  expected.  For  instance,  of  the  41 
Defense  Department  awards,  we  found  that 
the  technical  initiator  in  six  of  these  had 
recommended  on  his  procurement  request  that 
only  one  company  be  approached  to  undertake 
the  R&D  task.  (The  jargon  for  such  a  com- 
pany is  "desired  sole-source."  I  may  lapse  into 
the  jargon  now  and  again.)  Now,  acting  under 
the  procurement  principle  of  maxim)im  com- 
petition, the  government  agency  did  solicit 
other  firms  to  compete  for  these  awards;  An 
average  of  seven  firms  got  into  the  competi- 
tion on  each  of  these  desired  sole-source  proj- 
ects. But  who  got  the  awards?  After  all  these 
proposals  were  evaluated — usually  by  small 
teams,  dominated  by  the  government  techni- 
cal initiator — five  of  the  six  awards  went  to 
the  desired  sole-source  company  anyway.   (In 
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the  sixth  case — the  loser — this  desired  sole- 
amirce  was  severely  reprimanded  by  the  tech- 
nical initiator  for  having  "insulted  the  agency 
by  sending  in  an  'advertising  brochure.'  "  Evi- 
dently, even  a  company  that  is  "in"  can  some- 
times get  too  cocky.) 

In  nine  other  cases,  the  government  proj- 
ect initiator  recommended  two  or  three  com- 
panies in  each  case.  In  six  of  these  nine,  the 
award  went  to  one  of  those  recommended  com- 
panies— after  ten  to  forty  companies  had  been 
solicited  in  each  case. 

Now  we  have  covered  15  of  the  41  DOD 
awards.  What  about  the  other  26?  In  these 
cases,  the  initial  recommendation  lists — pre- 
pared by  the  project  initiators — contained 
more  than  three  company  names,  ranging  up 
to  21  suggested  sources.  It  was  no  longer  as 
easy  to  assert  that  a  narrow  solicitation  was 
desired  by  the  project  initiator.  But  we  dis- 
covered that  22  of  the  26  were  not  in  alpha- 
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betical  order.  Did  these  lists,  in  fact,  contain 
a  preference  indication?  Our  hypothesis 
claimed  that  if  the  project  initiator  had  his 
way — with  formal  procurement  regulations 
absent — he  would  award  the  contract  to  a 
firm  that  was  high  on  his  list. 

When  we  checked  our  hunch,  we  found  that 
10  of  these  22  awards  went  to  the  first  com- 
pany listed;  three  went  to  the  number  two 
firm ;  one  award  went  to  the  only  recom- 
mended company  that  bid ;  five  other  awards 
went  to  companies  the  initiators  had  recom- 
mended. So,  of  the  22  awards,  19  were  made 
to  firms  on  the  recommended  lists.  I  do  not 
say  this  is  "unfair,"  or  that  initiators  ought 
to  be  "more  objective."  I  will  come  to  the 
point  of  this  in  a  minute,  with  some  discus- 
sion of  "objective  evaluation,"  but  I  want  to 
point  out  some  additional  facts  before  winding 
up  this  discussion :  Over  twice  as  many  com- 
panies were  solicited,  in  all,  as  had  been  rec- 
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DISTRIBUTION  OF  CONTRACT  AWARDS 

BY  INFLUENCES  ON  THE  AWARD  DECISION 
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oinmended  by  the  technical  initiators;  in  three 
cases,  the  company  that  appeared  at  the  top 
of  the  list  did  not  even  bid  on  the  job;  in  only 
three  cases  did  the  award  go  to  a  firm  that 
had  not  been  recommended  by  the  project's 
initiator. 

But  what  about  the  four  lists  that  were 
made  alphabetically?  Again,  there  were  about 
twice  as  many  companies  solicited  as  were 
recommended  by  the  initiators,  but  in  all  four 
cases  the  award  went  to  a  firm  that  had  been 
recommended  by  the  initiator.  In  two  of  these 
four  cases,  we  found  from  interviews  that  the 
initiator  had  been  strongly  biased  in  favor  of 
the  ultimate  winner,  and  in  a  third  case  the 
contracting  officer  said  the  winner  could  have 
been  made  the  sole  source.  Thus,  it  is  possible 
that  alphabetical  source  lists  were  established 
merely  to  give  the  appearance  of  an  impartial 
recommendation. 

I  must  caution  that  these  data  were  drawn 
from  R&D  contracts  in  the  range  of  $100 
thousand  to  several  million  dollars  and,  fur- 
ther, that  all  of  these  41  awards  were  made 
by  the  same  agency  in  the  Defense  Depart- 
ment. Hence,  it  is  possible  that  less  "preselec- 
tion" may  take  place  in  other  agencies  and 
with  larger  contracts,  though  the  evidence  of 
Peck  and  Scherer  shows  a  very  high  percent- 
age of  formal  sole-source  awards  on  large- 
weapons  systems. 

But  with  R&D  contracts  of  less  than  $100 
thousand,  it  is  likely  that  even  more  preselec- 
tion occurs.  I  believe  this  because  fewer  for- 
mal reviews  are  required  and  decision-making 
authority  exists  at  a  lower  level  of  the  govern- 
mental organization.  I  intend  to  check  out 
this  hunch  in  the  near  future. 

Assuring    objective   evaluation 

The  second  key  principle  in  the  procure- 
ment regulations  is  that  award  decisions 
should  be  based  on  objective  evaluations  of 
the  proposals  that  are  submitted.  Following 
the  formal  procedure,  one  seeks  such  objec- 
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tivity  by  getting  quantitative  assessments 
from  several  competent  and  unbiased  evalua- 
tors.  But  our  research  results  show  that  many 
aspects  of  this  principle  are  denied. 

I  said  earlier  that  the  project  initiator  is 
often  biased  in  favor  of  one  or  two  companies 
—firms  with  whom  he  has  had  contact  and  ex- 
perience. Why  do  I  say  this?  The  project  ini- 
tiator is  always  a  technical  man.  And  any 
experienced  person  in  the  position  of  project 
initiator  will  ha\*e  had  some  previous  en- 
counters with  some  of  the  potential  bidders. 
It  is  hard  to  imagine  that  the  initiator's  judg- 
ment will  be  unaffected   by  such  experiences. 
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Recatise  he  is  almost  alwayn  a  member  of  the 
project's  evaluation  team,  the  initiator  can 
exercise  his  "predispositions."  Indeed,  one 
whose  jiidRment  in  unaffected  is  probably  not 
suitable  to  handle  evaluation  responsibilities. 
Hence,  I  would  say  two  things:  Not  only  is 
lack  of  some  bias  impossihle  of  attainment; 
it  is  probably  even  undesirable. 

Only  the  naive  nontechnical  man  can  believe 
that  technical  performance  and — more  diffi- 
cult— technical  proposals  can  be  "objectively" 
evaluated.  In  all  areas  worthy  of  the  "R&D" 
dosiRnation,  only  subjective  evaluation  is  pos- 
sible. Even  technical  "facts"  are  subject  to 
dispute  by  competent  evaluators.  Certainly, 
technical  "opinions"  on  yet  unproven  research 
and  development  projects  can  be  debated  still 
more.  We  can  expect  honest  appraisals  by 
competent  men,  but  these  must  reflect  their 
experiences,  judgments,  technical  prejudices, 
and  other  factors  of  a  subjective  nature. 
After-the-fact  contract  performance  evalua- 
tions in  the  contracts  investigated  often 
showed  that  award  decisions  had  been  based 
on  incorrect  technical  evaluations.  One  project 
initiator  admitted  that  he  had  been  "sold  a 
bill  of  goods  by  Company  X."  Another  proj- 
ect, awarded  to  Company  Y  because  of  its 
unique  technical  approach,  was  canceled  when 
the  approach  proved  unusable.  At  best  we  can 
expect  competent  subjectivity,  not  some  mysti- 
cal objectivity. 

What  about  using  numerical  point  scores  in 
evaluating  proposals?  In  nine  of  the  ten  non- 
defense  R&D  awards  of  over  $1  million,  the 
source  evaluation  board  members  revealed 
that  general  discussion,  leading  to  general 
agreement  on  the  recommended  source,  pre- 
rrdrd  final  assignment  of  numerical  point 
scores.  In  other  words,  the  numbers  are  not 
the  measures  that  produce  the  award;  rather 
they  are  after-the-fact  representations  of 
general  agreements.  They  are  justifications  for 
decisions,  rather  than  the  causes. 

I  challenge  other  aspects  of  the  objectivity 
principle.  For  example,  company  evaluation, 
not  propo.-^al  evaluation,  is  the  paramount  con- 
sideration. Furthermore,  two  or  three  key 
people,  at  most,  dominate  the  award  decisions 
— not  the  20  to  100  who  staff  all  evaluation 


committees.  One  ^ight  even  question  the  rela- 
tive competence  of  government  engineers  and 
scientists  to  evaluate  large  system  proposals, 
where  the  main  criteria  often  relate  to  organi- 
zation and  management  of  a  several  thou- 
sand man,  multicompany,  contractor  team. 

Independent  multilevel  review 

The  final  premise  of  the  formal  govern- 
ment R&D  contracting  system  is  that  several 
levels  of  review  and  approval  ensure  effective 
and  honest  award  decisions.  This  principle 
also  seems  more  an  illusion  than  a  reality. 

In  contracts  of  less  than  $1  million,  forma! 
source  evaluation  boards  are  seldom  estab- 
lished. An  award  recommendation  is  made  by 
the  technical  evaluation  group,  consisting  usu- 
ally of  the  technical  initiator,  his  supervisor, 
and  perhaps  another  scientist/engineer  in  the 
same  group.  Their  recommendation  goes  to 
the  contracting  officer,  a  member  of  the 
agency's  procurement  organization  who  rarely 
has  a  technical  background  or  technical  re- 
view capabilities.  He  looks  at  the  budget,  the 
relative  bids,  and  perhaps  at  some  recent 
agency  experiences  with  the  recommended 
contractor.  If  he  is  concerned  about  large  ap- 
parent cost  differences  in  the  bids,  he  might 
ask  the  technical  evaluators  if  a  lower  bidder 
would  be  acceptable.  But  so  long  as  budgeted 
funds  cover  the  recommended  bidder,  the 
technical  evaluators  can  have  their  way. 

For  the  larger  contracts,  technical  and  busi- 
ness evaluation  committees  report  to  a  source 
evaluation  board,  which  recommends  action 
to  the  center  director,  who  (in  very  large 
awards)  forwards  his  recommendation  to 
Washington  for  a  formal  decision.  But  do 
these  multiple  review  levels  mean  anything? 

Our  interviews  indicate  that  decision  mak- 
ing is  done — effectively,  if  not  formally — in 
the  technical  evaluation  committee  or  the 
source-evaluation  board.  In  some  cases,  the 
technical  initiator  "stacks"  the  board  with 
people  predisposed  to  accepting  the  initiator's 
recommendations.  Here,  the  real  power  re- 
sides with  the  technical  initiator,  acting 
through  his  technical-evaluation  committee; 
higher-level  groups  in  effect  "rubber  stamp" 
his  recommendations. 


Board  likes  you 

Mnvf?  nhr-rrrt  | 


Still  no  siqn  of 

DESIRED  SOLE 

SOURCE 


Board  change.s  mind 
Smile  of  nice  nurae 


Board  does 
like  you 

Move  ahead  1 


Keep  moving... 
You're  almost 
home! 


74      INTFRNAiinri*!.    ^rirMiF    Ann   XFrnNnior.v      Set^lrithrr    IQftl 


In  other  caaea,  we  found  thnt  the  source- 
evaluation  board  was  doing  the  decision  mak- 
inp,  with  the  supposedly  independent  commit- 
tees under  the  board  continuously  being 
called  upon  to  revise  their  reports  until  the 
senior  (rr'>iip  agreed  with  it.  As  one  procure- 
ment official  who  had  chaired  such  a  business- 
evaluation  committee  explained,  "They  tell  me 
that  when  I'm  on  an  evaluation  team,  I'm  sup- 
posed to  be  Joe  Independent.  What  they  forget 
is  that  for  the  rest  of  the  year  I'm  Joe  Sub- 
ordinate, and  if  my  boss  says  'Change  it,'  he's 
the  boss." 

In  only  one  case  of  the  ten  large  nondefense 
awards  was  the  source-evaluation  board's 
recommendation  reversed  at  the  Washington 
level.  The  situation  was  one  of  a  board  split 
between  two  competitors;  the  board's  dis- 
agreement resulted  from  different  interpreta- 
tions of  the  job  requirements.  Headquarters 
had  to  resolve  this  dispute,  but  in  no  other  case 
did  headquarters'  interference  or  review  ap- 
pear to  be  important. 

Peal  awaril  process 

The  real  award  process  is  one  involving 
long-term  person-to-person  contacts  between 
technical  people  in  government  and  industry. 
They  build  up  common  experiences,  attitudes, 
aspirations,  confidences.  And  ideas  are  gener- 
ated in  this  interchange.  These  are  the  ideas 
which  later  become  government-sponsored 
R&D  projects.  When  he  is  convinced  that  an 
idea  has  solid  merit,  the  government  scientist/ 
engineer  initiates  a  procurement  request.  He 
often  feels,  naturally,  that  the  work  should  be 
carried  out  by  the  people  in  whose  capabilities 
he  has  faith.  Acting  in  what  he  believes  to  be 
the  nation's  best  interest,  he  tries  to  secure 
"his"  contractor.  (He  usually  succeeds.)  If  he 
is  confident  of  his  judgment,  he  thwarts  at- 
tempts to  saddle  his  project  with  another  con- 
tractor. Only  when  he  regards  several  com- 
panies as  being  highly  qualified  does  real 
competition  prevail. 

I  do  not  mean  that  all  proposals  are  not 
carefully  studied  for  their  technical  content. 
They  are.  But  the  evaluatora  also  consider  the 
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companies:  their  experience,  their  key  people 
who  will  staff  the  job,  and  so  on.  What  the 
evaluators  must  really  look  at  are  the  contrac- 
tors as  potential  sources  of  work,  not  just  at 
their  glossy  brochures.  And  award  decisions 
are  reached  after  careful  consideration,  dis- 
cussion, and  argument  have  resolved  the  is- 
sues. Yes,  these  resolutions  are  quantified  in 
numerical  evaluation  forms— but,  as  I  say,  the 
numbers  follow,  not  precede,  the  point  of 
award  decision.  Occasionally,  a  unique  techni- 
cal proposal  or  a  large  cost  difference  in  the 
bids  upsets  the  expected  results.  But  far  more 
often,  the  predetermined  decision  proceeds 
unmolested  through  the  several  formal  stamps 
of  approval,  pausing  for  review  only  when  a 
major  unsettled  dispute  rears  its  head. 

Does  this  process  lend  itself  to  political 
pressure?  We  looked  for  this,  but  found  no 
instance  of  it.  And  I  doubt  that  our  interview 
subjects,  who  were  indeed  candid  on  other 
touchy  issues,  tried  to  hide  information  from 
us  on  this  question.  It  may  be  that  because  the 
awards  we  examined  were  fairly  small— 
mostly  less  than  $1  million — they  did  not 
merit  such  pressure.  But  Peck  and  Scherer's 
study,  of  bigger  contracts,  came  to  pretty 
much  the  same  conclusion:  "Political  consid- 
erations have  not  played  a  really  major  role 
in  the  choice  of  contractors  for  advanced 
weapons  programs."  They  found  a  lot  of  polit- 
ical "activity,"  but  concluded  that  the  effort 
did  not  have  a  corresponding  effect.  They  said 
that  much  of  the  activity  is  "ritualistic"  and 
that  political  pressures  tend  to  cancel  out  one 
another. 

In  only  one  of  the  large  (^  $1  million) 
awards  did  we  encounter  any  political  activity 
and  it  was  not  really  "pressure":  Two  Con- 
gressmen indicated  their  interest  in  the  con- 
tract; one  asked  that  two  competing  com- 
panies from  his  district  be  considered,  in  light 
of  high  unemployment  there;  the  other  Con- 
gressman, extolling  the  merits  of  small  busi- 
ness, gained  an  audience  with  the  center  di- 
rector for  the  president  of  a  small  company. 
The  agency  responded  by  tightening  up  inter- 
nal documentation  on  nil  phases  of  the  com- 
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petition  "in  case  of  any  trouble."  Two  of  the 
three  "Congressional"  companies  would  have 
been  disqualified  at  an  early  stage  of  the  eval- 
uation, but  this  was  not  done,  so  as  not  to  in- 
sult the  Congressmen.  (Both  companies  were 
ranked  near  the  bottom  in  the  final  evalua- 
tion.) The  company  that  did  receive  the  con- 
tract was  not  one  that  a  Congressman  had 
recommended.  When  the  award  was  an- 
nounced, the  Congressmen  were  notified  simul- 
taneously with  notification  to  the  bidders. 
Thus  our  only  incident  of  political  activity 
appears  as  appropriate  actions  by  Congress- 
men in  behalf  of  their  beliefs  and  constitu- 
ents; we  note  the  somewhat  ritualistic  flavor, 
the  partial  canceling  effect,  and  the  ultimate 
lack  of  influence  on  the  award  decision  itself. 

Policy  questions 

The  Department  of  Defense  under  Secre- 
tary McNamara  has  gone  in  heavily  for  the 
use  of  cost-effectiveness  analyses  in  evaluating 
weapons  systems.  Similarly,  I  believe  that  the 
research  and  development  cbntracting  policies 
of  the  Defense  Department,  and  other  govern- 
ment agencies,  would  be  strengthened  if  sim- 
ilar cost-effectiveness  criteria  were  applied. 

Let  us  examine  both  the  costs  and  benefits 
of  the  preprnt  system  of  R&D  contracting: 
We  now  know  that  the  real  process  underlying 
R&D  awards  is  informal  and  highly  selective. 
But  the  pressure  of  the  procurement  regula- 
tions and  fear  of  the  General  Accounting  Of- 
fice and  Congressional  inquiry  has  forced  us 
to  take  the  formal  solicitation  approach  to 
contractor  selection.  This  approach  is  bad  on 
two  counts:  It  is  costly  and  it  is  superficial. 
Let  me  substantiate  mv  argument,  first,  by 
citing  some  facts  that  others  have  uncovered. 

In  a  study  of  proposal  efforts  on  small  R&D 
tasks — from  $30  thousand  to  $150  thousand — 
Thomas  Allen  of  MIT  gathered  data  on  14 
contract  awards.  He  found  that  the  total  cost 
of  all  company-proposal  efforts  in  each  com- 
petition ranged  from  3%  to  150%  of  the  di- 
rect cost  of  the  contract  awarded.  The  total 
proposal  effort  in  those  14  competitions  was 
not  a  function  of  the  size  of  the  job,  but  was 
related,  rather,  to  the  number  of  bidders:  the 


more  bidders,  the  higher  the  total  proposal 
preparation  cost.  I  am  now  gathering  similar 
cost  data  on  the  contracts  in  my  sample. 

Another  study  tells  of  36  firms  competing 
for  a  single  large  R&D  contract:  Each  of  the 
36  bidders  spent  an  estimated  3000  to  4000 
engineering  man-hours  on  its  proposal.  Add 
these  together  and  you  have  45  to  60  man- 
years  of  engineering  effort.  And  these  figures 
represent  only  the  effort  expended  at  the 
prime  level.  Then  you  must  add  the  technical 
manpower  expended  by  various  tiers  of  sub- 
contractors, who  must  generate  technical  data 
for  the  prime  contractors,  the  bidders  them- 
selves. 

And  government  people  must  also  expend 
great  effort,  what  with  extensive  source  so- 
licitation and  the  ritual  of  "going  through  the 
motions"  on  the  proposals  submitted.  The  key 
people  on  technical  staffs  often  serve  for  weeks 
and  months  ori  the  evaluation  teams,  distracted 
from  project  Work  in  which  they  might  other- 
wise be  enga'ged.        ' 

The  direct  financial  cost  of  this  process  is 
high,  and  governments  bears  the  entire  cost  of 
the  evaluatioft  and  most  of  the  cost  (through 
allowed  contract  overheads)  of  the  company 
proposal  preparation;  More  important  than 
this  direct  cost  is  the  resulting  reallocation  of 
people:  The  best  technical  personnel,  in  in- 
dustry and  government,  are  pulled  off  into 
proposal  preparation  and  evaluation.  This  de- 
tracts immeasurably  from  progress  and  ef- 
fectiveness of  the  ongoing  projects.  It  estab- 
lishes a  situation  in  which  the  best  industry 
people  propose  and  try  to  sell  the  jobs,  while 
the  mediocre  are  left  to  do  the  work.  Further- 
more, significant  time  delays  are  added  to  the 
contracting  process  by  the  necessity  of  the  3 
to  12  months  needed  to  prepare  proposal  re- 
quests, to  solicit  proposals,  to  evaluate  them, 
and,  finally,  to  make  the  awards. 

These  factors  do  not  contribute  to  the  ulti- 
mate success  of  R&D  projects,  yet  current 
government  pressures  are  pushing  for  an  ex- 
pansion of  formal  competition,  and  for  an  in- 
crease in  the  number  of  companies  solicited 
for  each  project.  This  tendency  will  increase 
still  further  the  direct  and  indirect  costs  of 
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R&D  procurement  to  both  government  and  in- 
dustry, and  will  draw  even  more  engineers 
away  from  contract  work  toward  proposal  ef- 
forts. 

Benefits  o/  R&D  procurement  regulations 

How  effective  is  the  current  regulatory 
system?  Though  hard  to  measure,  it  scores 
positively  on  several  counts:  For  instance, 
one  index  of  effectiveness  might  be  the  num- 
ber of  "surprises"  resulting  from  the  en- 
forced broad  solicitations.  We  saw  earlier  that 
seven  of  the  41  DOD  contracts  were  awarded 
to  companies  not  on  the  initiator's  recom- 
mended list;  five  others  went  to  firms  that  had 
been  recommended,  but  not  necessarily  pre- 
ferred, by  the  initiator.  Now  some  of  these 
12  proposals  were  more  outstanding  than 
those  submitted  by  preferred  firms.  These 
were  the  surprises.  And  this  is  one  benefit  of 
the  current  system :  Broad  solicitation  occasion- 
ally uncovers  someone  with  a  better  approach 
to  the  problem  solution. 

But  note  that  at  least  four  of  these  twelve 
awards  were  made  because  of  apparent  cost 
differences  in  the  proposals,  not  because  of 
better  technical  approaches.  And  in  two  of 
these  four  cases,  it  turned  out  that  the  as- 
sumed cost  benefits  had  been  grossly  mislead- 
ing, and  the  government  project  monitors  re- 
gretted the  awards.  Hence,  we  can  say  that 
cost  savings  do  result  in  some  instances,  due 
to  the  present  system  of  widespread  solicita- 
tion. But  this  benefit  cannot  be  evaluated  with 
precision. 

A  final  assumed  benefit  of  the  existing  reg- 
ulations is  the  built-in  system  of  checks  to 
ensure  honesty  in  contractor  selection.  This 
is  intended  to  prevent  mishandling  of  govern- 
ment funds  for  research  and  development.  I 
will  speak  to  this  point  in  a  minute. 

How   to   improve   the   system 

Do  these  benefits  justify  the  burdens?  I 
think  not.  Particularly  with  awards  of  under 
$1  million,  I  believe  the  costs  of  the  regula- 
tions far  outweigh  the  rewards.  The  misallo- 
cation  of  talent  and  the  added  time  delays  are 
more  disturbing  here  than  are  the  large,  di- 
rect proposal  costs.  Further,  the  internal 
checking  provided  at  present  is  both  ineffec- 
tive (as  shown  by  the  high  degree  of  preselec- 
tion) and  unnecessary.  I  am  convinced,  from 
our  interviews  with  project  initiators  and 
contracting  officials,  that  these  people  are 
highly  motivated  "to  get  the  most  for  Uncle 
Sam."  The  honesty  of  these  government  scien- 
tists and  engineers  is  unquestionably  high; 
their  ethical  standards  have  not  been  cor- 
rupted by  the  intentional-low-bidding  game 
that  current  procurement  regulations  en- 
courage industry  to  play.  It  seems  foolhardy 
to  continue  buying  insurance  when  the  pre- 
mium is  more  costly  than  the  value  of  what  is 
being  insured. 


Precedents  already  exist  that  indicate  a 
path  for  improving  the  R&D  award  system. 
For  example,  the  Atomic  Energy  Commission 
has  visited  company  plants  when  determining 
the  award  of  a  contract.  The  Army  also  has 
relied  heavily  on  such  visit-interviews  in  eval- 
uating contractors  for  its  Sprint  program. 
Indeed,  DOD  regulations  permit  the  use  of 
oral  proposals  as  a  means  for  source  selection; 
formal  encouragement  to  apply  this  approach 
was  provided  recently  in  a  memo  from  As- 
sistant Secretaries  Fubini  and  Morris  of  the 
DOD.  Other  informal  or  less  formal  means 
of  industry  solicitation  and  evaluation  are 
possible  and  can  be  used  by  technical  evalua- 
tors. 

For  R&D  awards  under  $1  million,  techni- 
cal initiators  should  have  official  authority 
and  real  backing  to  solicit  industry  informally 
and  to  select  sources  without  the  requirement 
of  openly  solicited  written  proposals.  The  ini- 
tiator, if  he  wishes,  should  be  permitted  to 
use  any  mix  of  methods,  including  oral  pro- 
posals, plant  visits,  restricted  solicitation,  as 
vvell  as  the  broad  solicitation  approach  cur- 
rently used.  In  turn,  the  initiator  and  the 
other  evaluators  should  be  required,  after  the 
fact,  to  write  up  and  justify  their  method  of 
evaluation,  including  a  written  assessment  of 
the  companies  considered.  Numerical  evalua- 
tion should  not  be  required,  though  it  could 
be  used  if  desired  by  the  evaluation  team.  I 
make  these  suggestions  in  order  to  encourage 
more  flexibility  in  source  selection  methods 
and  to  place  greater  trust  and  responsibility 
in  the  government  evaluators.  I  believe  these 
changes  are  in  the  direction  in  which  the 
entire  government  contracting  system  should 
move. 

At  all  levels  of  R&D  contracting  (^^  and  < 
$1  million)  a  criterion  for  the  source  selection 
approach  used  should  be  that  the  "costs  to 
procure"  should  relate  to  the  costs  of  the  pro- 
curement itself,  and  to  the  added  value  be- 
lieved achievable  by  the  selection  method.  For 
example,  we  now  ask  as  many  as  100  com- 
panies to  submit  proposals  on  R&D  contracts 
valued  at  under  $100  thousand.  This  practice 
is  foolish  and  should  be  abandoned.  Courage 
is  needed  in  the  DOD,  NASA,  and  other 
agencies  to  resist  the  current  trend  to  write 
cost-savings  reports  based  on  the  number  of 
contracts  written  "competitively,"  instead  of 
as  sole-source  awards.  Our  evidence  indicates 
that  total  costs  may  well  be  increased,  rather 
than  reduced,  by  such  a  forced  changeover  to 
superficial  competition.  Instead  of  attempting 
cost  reduction  in  research  and  development, 
far  more  gain  is  possible  by  striving  for  ef- 
fectiveness increases,  and  these  increases  are 
only  possible  by  ridding  ourselves  of  the  con- 
straints of  the  current  formal  regulations. 

For  more  information  on  the  processes  of 
governmental  R&D  contract  awards,  see  the 
references  on  p.  103. 
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